Method of analyzing multiple range test data

ABSTRACT

A method and graph for analyzing the performance of a testing device that operates over at least two ranges, such as a hematology testing device. The method of the invention comprises forming generally parallel data range axes, each relating to a range of operation of the testing device, and, preferably, labeling at least one of the data range axes with standard deviation index units. The method further comprises obtaining data from a testing device to be analyzed and from a group of testing devices that operate over at least two ranges. Individual ranges for each of at least two ranges as well as group ranges and group standard deviations for at least two ranges are calculated from these test results. In the preferred embodiment, standard deviation indexes are computed for the testing device based on these calculations. The standard deviation indexes are plotted, and the resulting plot is preferably used to analyze the performance of and calibrate the testing device. Similarly, the graph of the invention comprises a graph with at least two generally parallel data range axes and a plot of deviation indexes for the testing device under analysis.

FIELD OF THE INVENTION

This invention relates to a method of analyzing test data, and more particularly, to a method of analyzing test data of equipment having a multilevel control system to thereby determine if the equipment is operating properly.

BACKGROUND OF THE INVENTION

Hematology equipment is used to analyze the blood of a patient. As with most equipment, it is necessary to calibrate the hematology equipment to obtain accurate tests results. The need to calibrate hematology equipment has become increasingly important due to recent legislative enactments which have required more frequent monitoring of the equipment to ensure proper calibration. The heightened legislative attention to accurate hematology equipment performance is due, no doubt, to the substantial escalation of severe transmittable blood diseases, and in particular, the AIDS virus. The more stringent calibration requirements are therefore certainly prudent, but nevertheless have significantly increased the calibration burden for users of laboratory equipment.

FIG. 1 shows three prior art graphs used for analyzing the operation of a hematology testing device. The graphs of FIG. 1 are commonly referred to as Levey-Jennings plots. As noted at the top of each graph in FIG. 1, each graph depicts data from a particular "assayed range." Depending on the age of the patient, the test results produced by a particular hematology testing device vary significantly in value. A hematology testing device must therefore produce accurate tests results over multiple ranges, and thus, must be calibrated to ensure proper operation over each range.

Referring to FIG. 1(a), the graph depicts the test results of a particular hematology testing device over the "assayed range" of 18 to 24. The various test points, such as point 10, represent particular test results from patient blood analyses taken by a particular hematology device. FIG. 2 shows a table summarizing the results of the data for the graphs of FIG. 1. The mean for the data given in FIG. 1(a) is 20.92 and the standard deviation is 1.13. The table of FIG. 2 also includes data relating to results for a group of hematology devices so that the results of the particular hematology device can be compared to the performance of the group of hematology devices as a whole. The average performance of a group of hematology devices is believed to be an effective method for evaluating the performance of a particular hematology device. If the particular device deviates significantly from the average performance characteristics of the group, this result indicates a problem with the performance of the particular device. Likewise, if the performance of the particular device substantially corresponds with the average performance of the group, this result indicates that the particular device is operating properly and does not require calibration.

The graphs of FIG. 1 have three regions 12, 14, and 16, graphically depicted by shading, and being centered around the line 17 corresponding to the mean for the data of the group of hematology testing devices over the particular level or assayed range. The dark shading indicates data in an unacceptable range; the region without shading indicates data in an acceptable range; and the slightly shaded region indicates data that is borderline, and therefore may or may not be acceptable. The shading is particularly helpful because it provides a readily identifiable visual indicator of the significance of each data point. In short, the graphs of FIG. 1 and the table of FIG. 2 facilitate calibration of a particular hematology testing device. Nevertheless, analyzing the performance of a hematology device required analysis of multiple graphs (i.e., one graph for each data range).

In an effort to allow multiple hematology devices to be analyzed at the same time, another prior art graph was developed (shown in FIG. 3) and is commonly referred to as a Youden Plot. The Youden Plot allows multiple hematology testing devices to be analyzed over two assayed ranges, and was thus a meaningful advance in the art of calibrating hematology testing devices. Referring to FIG. 3, the vertical axis of the graph is centered about the mean of the group data for the assayed range of 18 to 24 (i.e., 21.6), and the horizontal axis is centered about the mean of the group data for the assayed range of 46 to 58 (i.e., 52.9). Point 20 corresponds to the data point for the hematology testing device data from the graphs of FIG. 1(a)&(b) and the table of FIG. 2. The horizontal darkened line 22 corresponds to the mean of the data (i.e., 20.92) of the hematology device over the range of 18 to 24, and the vertical darkened line 24 corresponds to the mean of the data (i.e., 51.92) of the hematology device over the range of 46 to 58. As shown, the data point 20 is in the unshaded region for both the horizontal and vertical axes, and thus demonstrates that the mean for the hematology device over both ranges is acceptable. The remaining data points, such as points 30, 32, and 34, correspond to the mean values of other hematology devices over the ranges of 18 to 24 and 46 to 58.

A significant limitation of the Youden Plot is that only two ranges of data can be analyzed. As explained above, hematology devices operate over multiple ranges to accommodate varying data ranges of blood test results due to such variables as the varying ages of patients. Thus, to completely analyze and calibrate a particular hematology device, several graphs typically must be analyzed. A further limitation of the Youden Plot is that the horizontal and vertical axes are obviously perpendicular, and thus it is very difficult to analyze the linear performance of the hematology device over both ranges depicted on the plot. Each of these limitations are, likewise, common to the Levey-Jennings plots.

While the limitations above have been described in the context of a hematology device, the limitations apply to all devices having multilevel control systems, or stated another way, devices which operate over multiple ranges. The majority of body fluid chemistry analysis devices implement multilevel control systems, and thus suffer from the above-described limitations.

OBJECT AND SUMMARY OF THE INVENTION

It is therefore an object of the present invention to provide a method for improving and facilitating the analysis of the performance of equipment or devices which have multilevel control systems. Another object is to provide a method which allows for efficient and effective analysis of such equipment over all operational ranges of the equipment to alleviate the need to analyze multiple graphs to calibrate the equipment.

And yet another object is to provide a method for graphing the test results of such equipment which facilitates analysis of the linear operation of the equipment over all ranges of operation of the equipment. A related object is to provide a method of graphing the test results of the equipment performance to allow persons to efficiently and accurately analyze the results to determine if the equipment is operating properly without the need for sophisticated training.

A further object is to disclose a graph and a method of using such a graph which facilitates analysis of devices operating over multiple ranges, and which method of graphing and analyzing is also more efficient and less time consuming than prior art analytical methods.

To accomplish these and related objects, a method is disclosed for analyzing data relating to at least two ranges. The method comprises forming generally parallel spaced apart data range axes for each range, and forming an axis in generally orthogonal relation to the data range axes. In the preferred embodiment, the method further comprises labeling the data range axes with standard deviation index (SDI) units.

In applying the method to hematology devices, the method further comprises gathering data from a group of hematology devices that operate over at least two ranges. The method further comprises calculating the mean for data gathered for each range for each hematology device, calculating the mean for data gathered for each range of the group of hematology devices, calculating the standard deviation for the data gathered for each range for the group of hematology devices, and calculating the standard deviation index for data gathered for each range of each of the hematology devices. The method also comprises plotting the standard deviation index data for each hematology device on the graph, thereby allowing analysis of the linear performance of each hematology device over at least two ranges, and more specifically, over all ranges of operation for each hematology device.

The analysis method described for hematology devices applies equally as well to any device having a multilevel control system. The above-described method overcomes the problems of the prior art by allowing analysis of all operational ranges of devices having multilevel control systems. The method also significantly facilitates analysis of the linear performance of such devices over all operational ranges. Moreover, the method is more efficient than the prior art analysis methods because only one graph is required to perform calibration analysis over all operational ranges, in contrast to the multiple graphs required by prior art methods to calibrate multilevel control system devices.

BRIEF DESCRIPTION OF THE DRAWING

In the accompanying drawings which form a part of the specification and are to be read in conjunction therewith, and in which like reference numerals are used to indicate like parts in the various views:

FIGS. 1A-1C show three prior art graphs for data of a device having a multilevel control system, and each graph displays data corresponding to a particular range of operation of the device;

FIG. 2 is a table of the data plotted for each of the graphs in FIG. 1;

FIG. 3 is a prior art Youden plot displaying data for two ranges of performance of the device;

FIG. 4 shows a preferred embodiment of the graph of the present invention used in the method for analyzing the performance of a multilevel control system device;

FIG. 5 shows the method step of plotting data from a multilevel control system device on the graph of FIG. 4;

FIG. 6 shows the method step of plotting data from a multilevel control system device on the graph of FIG. 4;

FIG. 7 shows the method step of plotting data from a multilevel control system device on the graph of FIG. 4;

FIG. 8 shows the method step of plotting data from a multilevel control system device on the graph of FIG. 4; and

FIG. 9 shows the method step of plotting data from a group of multilevel control system devices on the graph of FIG. 4.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring now to FIG. 4, the graph required for implementing the method of the present invention is designated generally 60. The darkened line 62 is a plot of the data contained in the table of FIG. 2. As described above, the SDI data of that table relates to the operation of a device which operates over at least two assayed ranges, and for purposes of illustration, those ranges comprise: 18 to 24 , 46 to 58, and 72 to 94. Referring to FIG. 2, Roman numerals are used to designate corresponding data ranges and are designated by Arabic numeral 64.

The graph 60 of FIG. 4 is created by forming generally parallel spaced apart data range axes 66 for each range of data. Graph 60 comprises three data range axes 66, one axis corresponding to each of the data ranges of the table in FIG. 2. Roman numerals 64 corresponding to each data range axis 66 correlate to the Roman numerals associated with each data range in the table of FIG. 2. The method of creating the graph 60 further comprises forming an axis 68 in generally orthogonal relation to the data range axes 66.

In the preferred embodiment, the data range axes are labeled with standard deviation index (SDI) units. To calculate the SDI for each range of data for a device which operates over at least two ranges, such as a hematology device, the following data must be gathered: (1) mean of the data for each range for each hematology device; (2) mean for each range for the group of hematology devices; and (3) standard deviation for each range for the group of hematology devices. The SDI for each range for each hematology device is calculated according to the following equation:

(particular hematology device range mean-group range mean)/(group range standard deviation)

For instance, referring to the table of FIG. 2, the implementation of the equation to calculate the SDI for range I is as follows: ##EQU1##

As is well known in the art, the mean is a measure of the overall tendency of the data group, also known as the average. The standard deviation is a measure of the average distance that particular data points in the group deviate from the mean. The SDI, as is apparent from the equation, measures how far away the particular device mean is from the group mean in terms of the group standard deviation. Unitizing the difference between the group mean and the particular device mean by group standard deviation allows for comparison of the data for a particular device to the data of the group. An SDI of +1 results from a testing system mean being one group standard deviation above the group mean. An SDI of 0 results from a testing system mean matching the group mean, and an SDI of -1 results from a testing system mean being one group standard deviation below the group mean. A testing system mean which is two standard deviations away from the group mean (i.e., either an SDI of +2 or -2) is typically considered abnormal, thus evidencing a problem with the operation of the device. An SDI of +1 or -1 presents a borderline reading which may be problematic, but not critically so.

The regions between plus or minus 2 SDIs and plus or minus 3 SDIs, respectively, are designated 70 in FIG. 4. Visible indicia is provided within the regions 70, preferably, in the form of a dark shading. The regions between the plus or minus 1 SDI and the plus or minus 2 SDI, respectively, are designated 72. Region 72 is marked with a unique visible indicia, which in the preferred embodiment is a shading lighter than that of regions 70. The regions between 0 SDI and plus or minus 1 SDI are designated 74. The regions 74 are provided with a unique visible indicia, which preferably comprises a shading lighter than that of regions 70 and 72, and more specifically, preferably comprises no shading, as shown in FIG. 4.

FIGS. 5-7 are particular plots of data on the graph 60. The plot in FIG. 5 demonstrates a plot of a device which matched the group device mean for each data range. The plot in FIG. 6 demonstrates the plot of a device which is recovering control values consistently higher than the group mean. All three data ranges are running almost one group SDI above the group mean. This indicates a significant positive bias in the testing device which should be evaluated for cause. The plot in FIG. 7 demonstrates a plot for a device which is recovering higher control values than the group at range I, the same control values as the group at range II, and lower control levels than the group at range III. This plot indicates a linearity problem with the testing system because the testing device behaves differently at different operational ranges. The plot of FIG. 8 demonstrates the plot of a testing device with a different type of linearity problem. The device is close to the group mean at ranges I and III, but is recovering significantly higher control values than the group mean at range II.

In analyzing the graph 60, there are two important considerations: (1) the angle of the line, and (2) the region designated by shading in which the line lies. The angle of the line is a measure of the device's linear performance. A bent or sloping line indicates that the device is not behaving the same across different ranges of performance. As explained above, the dark shaded, light shaded, and nonshaded regions, designated 70, 72, and 74, respectively, correspond to problem action levels, possible problem action levels, and no problems, respectively.

A straight line down in the dark shaded region, such as regions 70, indicates a significant bias in the testing device. If the plot is angular with some of the points (or all) falling in the shaded regions, the degree of shading indicates the significance of the linearity distortion.

Referring to FIG. 9, an additional graph 60 is shown in which the performance of a particular device is designated by the thick line 83, and the performance of the other devices of the group are indicated by the thin lines, such as lines 84 and 86. The graph of FIG. 9 allows the performance of a particular hematology device to be readily compared to the performance of all hematology devices of the group.

To plot the data on the graph shown in FIG. 4, the following computer program can be implemented: ##SPC1##

The program is written for BORLAND C++, Version 3.1. The program reads from a data file containing the data points to be plotted.

After a graph, such as one of the graphs shown in FIGS. 4-9, is generated, the performance of the multilevel control system device can be analyzed over all operational ranges. The device being analyzed can thereafter be calibrated to correct any performance error that might be shown on the graph. Such calibration techniques vary depending on the particular device being analyzed. Nevertheless, such calibration techniques are well known in the art.

Thus, it can be seen that the graph 60 of the present invention and the method of plotting and analyzing data for multilevel control system devices thereon facilitate analysis of the linear performance of one device or multiple devices over all ranges of operation. It should be understood that a data range axis 66 is formed on the graph 60 for each range of performance for a particular device, and thus a plurality of data range axes corresponding to each range of performance of a device can be provided so that analysis of the performance of a device can be performed for every range of performance of the device.

From the foregoing, it will be seen that this invention is one well-adapted to attain all the ends and objects hereinabove set forth together with other advantages which are obvious and which are inherent to the structure.

It will be understood that certain features and subcombinations are of utility and may be employed without reference to other features and subcombinations. This is contemplated by and is within the scope of the claims.

Since many possible embodiments may be made of the invention without departing from the scope thereof, it is to be understood that all matter herein set forth or shown in the accompanying drawings is to be interpreted as illustrative and not in a limiting sense. 

What is claimed is:
 1. A method for analyzing the performance of a testing device for analyzing human bodily fluids which operates over at least two ranges, the method comprising:obtaining a first set of test results relating to human bodily fluids from the testing device over at least two ranges; calculating from the first set of test results an individual range mean for each of the at least two ranges; obtaining a second set of test results relating to human bodily fluids from a group of testing devices that operate over the at least two ranges; calculating from the second set of test results a group range mean and a group range standard deviation for each of the at least two ranges; calculating standard deviation indexes for the testing device from the individual range means, the group range means and the group range standard deviations; forming generally parallel spaced apart data range axes, each relating to a range of operation of the testing device, to facilitate analysis of the performance of the testing device over each range of operation; and plotting all of the standard deviation indexes in relation to the data range axes in such a way that analysis of the performance of the testing device over the at least two operating ranges is provided in a single graphic display.
 2. The method of claim 2 further comprising labeling at least one of the data range axes with standard deviation index (SDI) units.
 3. The method of claim 2 further comprising labeling a zero reference point along at least one of the data range axes, and labeling in one direction along at least one of the data range axes with positive increments of the SDI units, and labeling in the other direction along at least one of the data range axes with negative increments of the SDI units.
 4. The method of claim 3 further comprising labeling the zero reference point with 0, labeling the positive increments with +1 and +2 SDI units, and labeling the negative increments with -1 and -2 SDI units so that regions are created between adjacent SDI units.
 5. The method of claim 4 further comprising forming a first visible indicia within the regions between 0 and +1 or -1 SDI units, forming a second visible indicia within the regions between +1 or -1 SDI units and +2 or -2 SDI units, respectively, and forming a third visible indicia within the regions beyond the +2 or -2 SDI units, to uniquely identify the regions.
 6. The method of claim 5 wherein the first, second, and third visible indicia comprise different shadings.
 7. The method of claim 1 wherein the step of plotting comprises plotting the data using a computer program.
 8. The method of claim 7 further comprising analyzing the plotted data, and calibrating the testing device based upon the analysis.
 9. The method of claim 1, wherein the testing device comprises a hematology testing device.
 10. The method of claim 1, further comprising forming an axis in generally orthogonal relation to the data range axes.
 11. The method of claim 1 further comprising analyzing the plotted data, and calibrating the testing device based upon the analysis.
 12. A graph for analyzing the performance of a testing device for analyzing human bodily fluids which operates over at least two ranges, wherein the graph is formed using computer program, the graph comprising:at least two generally parallel spaced apart data range axes each relating to a range of operation of the testing device for facilitating analysis of the performance of the testing device over each range of operation; and a plot of deviation indexes formed by the computer program for the testing device along the data range axes, wherein the graph provides analysis of the performance of the testing device over each range of operation in a single graphic display.
 13. The graph of claim 12, wherein said graph further comprises an axis in generally orthogonal relation to the data range axes.
 14. The graph of claim 12, wherein said deviation indexes comprise standard deviation indexes.
 15. The graph of claim 14 further comprising standard deviation index (SDI) unit indicia for at least one of the data ranges axes.
 16. The graph of claim 15 wherein the data range axes comprise a midpoint and the SDI unit indicia comprise zero SDI unit indicia corresponding to the midpoint, positive increments of the SDI units beyond the midpoint on one side of the data range axes, and negative increments of the SDI units beyond the midpoint on the other side of the data range axes.
 17. The graph of claim 16 wherein the positive increments comprises +1 and +2 SDI unit indicia and the negative increments comprises -1 and -2 indicia such that regions are formed between adjacent SDI unit indicia.
 18. The graph of claim 17 further comprising a first visible indicia within the regions between 0 and +1 or -1 SDI units, a second visible indicia within the regions between +1 or -1 SDI units and +2 or -2 SDI units, respectively, and a third visible indicia within the regions beyond the +2 or -2 SDI units, to uniquely identify the regions.
 19. The graph of claim 18, wherein the first, second, and third indicia comprise different shadings.
 20. The graph of claim 12, wherein said testing device comprises a hematology testing device.
 21. A method for analyzing the performance of a testing device for analyzing biochemical samples which operates over at least two ranges, the method comprising:obtaining a first set of test results relating to a biochemical sample from the testing device over at least two ranges; obtaining a second set of test results relating to the biochemical sample from a group of testing devices that operate over the at least two ranges; calculating deviation indexes for the testing device from the first set of test results and the second set of test results by determining the deviation of the performance of the testing device from that of the group of testing devices for each of the least two ranges and unitizing the deviations; forming generally parallel spaced apart data range axes, each relating to a range of operation of the testing device, to facilitate analysis of the performance of the testing device over each range of operation; and plotting all of the deviation indexes in relation to the data range axes in such a way that analysis of the performance of the testing device over the at least two operation ranges is provided in a single graphic display.
 22. The method of claim 21 further comprising labeling a zero reference point along at least one of the data range axes, and labeling in one direction along the data range axes with positive increments of the deviation index units, and labeling in the other direction along at least one of the data range axes with negative increments of the deviation index units.
 23. The method of claim 21, wherein the testing device comprises a device for analyzing human bodily fluids.
 24. The method of claim 21, wherein the testing device comprise a hematology testing device. 